Heuristic Planning in Adversarial Dynamic Domains

Agents in highly dynamic adversarial domains, such as RTS games, must continually make time-critical decisions to adapt their behaviour to the changing environment. RTS games involve two players who build structures, recruit armies and fight for space and resources in order to control strategic points, destroy the opposing force and ultimately win the game. Other examples of adversarial domains include security and defense applications. In such a context, the planning agent must consider his opponent’s actions as uncontrollable, or at best influenceable. Turn-taking domains, such as chess, enjoy heuristic search methods which somewhat explicitly take into account the potential actions of the opponent to guide the search algorithm. However, for more general nondeterministic domains where there is no clear turn-taking protocol, most heuristic search methods to date do not explicitly reason about the opponent’s actions when guiding the state space exploration towards goal or high-reward states. In contrast, we are investigating a domain-independent heuristic planning approach which reasons about the dynamics and uncontrollability of the opponent’s behaviours in order to provide better guidance to the search process of the planner. Our planner takes as input the opponent’s behaviours recognized by a plan recognition module and uses them to identify opponent’s actions that lead to low-utility projected states. It relies on a planning graph (Ghallab, Nau, and Traverso 2004) to determine the agent’s actions that are likely to void the preconditions of these undesirable opponent actions or counter their effects. We believe such explicit heuristic reasoning about the potential behaviours of the opponent is crucial when planning in adversarial domains, yet is missing in today’s planning approaches.